In defense of a "grammar" in the visual language of comics

Visual Language Theory (VLT) argues that the structure of drawn images is guided by similar cognitive principles as language, foremost a "narrative grammar" that guides the ways in which sequences of images convey meaning. Recent works have critiqued this linguistic orientation, such as Bateman and Wildfeuer's (2014) arguments that a grammar for sequential images is unnecessary. They assert that the notion of a grammar governing sequential images is problematic, and that the same information can be captured in a "discourse" based approach that dynamically updates meaningful information across juxtaposed images. This paper reviews these assertions, addresses their critiques about a grammar of sequential images, and then details the shortcomings of their own claims. Such discussion is directly grounded in the empirical evidence about how people comprehend sequences of images. In doing so, it reviews the assumptions and basic principles of the narrative grammar of the visual language used in comics, and it aims to demonstrate the empirical standards by which theories of comics' structure should adhere to. (C) 2018 Elsevier B.V. All rights reserved

Meaning above (and in) the head: Combinatorial visual morphology from comics and emoji

AbstractCompositionality is a primary feature of language, but graphics can also create combinatorial meaning, like with items above faces (e.g., lightbulbs to mean inspiration). We posit that these “upfixes” (i.e., upwards affixes) involve a productive schema enabling both stored and novel face–upfix dyads. In two experiments, participants viewed either conventional (e.g., lightbulb) or unconventional (e.g., clover-leaves) upfixes with faces which either matched (e.g., lightbulb/smile) or mismatched (e.g., lightbulb/frown). In Experiment 1, matching dyads sponsored higher comprehensibility ratings and faster response times, modulated by conventionality. In Experiment 2, event-related brain potentials (ERPs) revealed conventional upfixes, regardless of matching, evoked larger N250s, indicating perceptual expertise, but mismatching and unconventional dyads elicited larger semantic processing costs (N400) than conventional-matching dyads. Yet mismatches evoked a late negativity, suggesting congruent novel dyads remained construable compared with violations. These results support that combinatorial graphics involve a constrained productive schema, similar to the lexicon of language.</jats:p

Zooming in on the cognitive neuroscience of visual narrative.

Visual narratives like comics and films often shift between showing full scenes and close, zoomed-in viewpoints. These zooms are similar to the "spotlight of attention" cast across a visual scene in perception. We here measured ERPs to visual narratives (comic strips) that used zoomed-in and full-scene panels either throughout the whole sequence context or at specific critical panels. Zoomed-in panels were automatically generated on the basis of fixations from prior participants' eye movements to the crucial content of panels (Foulsham & Cohn, 2020). We found that these fixation panels evoked a smaller N300 than full-scenes, indicative of reduced cost for object identification, but that they also evoked a slightly larger amplitude N400 response, suggesting a greater cost for accessing semantic memory with constrained content. Panels in sequences where fixation panels persisted across all positions of the sequence also evoked larger posterior P600s, implying that constrained views required more updating or revision processes throughout the sequence. Altogether, these findings suggest that constraining a visual scene to its crucial parts triggers various processes related not only to the density of its information but also to its integration into a sequential context

Preearthquake and Postearthquake Creep on the Imperial Fault and the Brawley Fault Zone

Taken together, 12 years of alinement-array data, 4 years of creepmeter records from four instruments, and 2 years of surveys from two nail files suggests that creep events on the Imperial fault 2 to 5 months before the October 15 earthquake are consistent with longterm trends and not indicative of any imminent event. No discernible creep occurred on the fault in the hours and days before the earthquake. Records of coseismic displacement imply that response of the soil to the fault slip at depth was brittle rather than plastic; they uniquely demonstrate that the minimum rate of surface fault displacement was 1.8 cm/s. Continuing measurements of afterslip show that all motion is due to discrete 0.2- to 1.5-cm creep events occurring less frequently over time. The accumulating displacement for the first 35 days after the earthquake is well approximated by linear logarithmic functions of time. Use of this accumulating displacement to predict future slip rates implies that for 6 years the afterslip rate from the 1979 earthquake should be greater than the 0.5-cm/yr average preearthquake creep rate. The maximum amount of slip on the surface trace of the Imperial fault associated with the 1979 earthquake, including afterslip, amounts to more than 60 cm

Visual and linguistic narrative comprehension in autism spectrum disorders:Neural evidence for modality-independent impairments

Individuals with autism spectrum disorders (ASD) have notable language difficulties, including with understanding narratives. However, most narrative comprehension studies have used written or spoken narratives, making it unclear whether narrative difficulties stem from language impairments or more global impairments in the kinds of general cognitive processes (such as understanding meaning and structural sequencing) that are involved in narrative comprehension. Using event-related potentials (ERPs), we directly compared semantic comprehension of linguistic narratives (short sentences) and visual narratives (comic panels) in adults with ASD and typically-developing (TD) adults. Compared to the TD group, the ASD group showed reduced N400 effects for both linguistic and visual narratives, suggesting comprehension impairments for both types of narratives and thereby implicating a more domain-general impairment. Based on these results, we propose that individuals with ASD use a more bottom-up style of processing during narrative comprehension

Cosmological Perturbations from the No Boundary Euclidean Path Integral

We compute, from first principles, the quantum fluctuations about instanton saddle points of the Euclidean path integral for Einstein gravity coupled to a scalar field. The Euclidean two-point correlator is analytically continued into the Lorentzian region where it describes the quantum mechanical vacuum fluctuations in the state described by no boundary proposal initial conditions. We concentrate on the density perturbations in open inflationary universes produced from cosmological instantons, describing the differences between non-singular Coleman-De Luccia and singular Hawking-Turok instantons. We show how the Euclidean path integral uniquely specifies the fluctuations in both cases.Comment: 21 pages, RevTex file, including five postscript figure file

A multi-layer extension of the stochastic heat equation

Motivated by recent developments on solvable directed polymer models, we define a 'multi-layer' extension of the stochastic heat equation involving non-intersecting Brownian motions.Comment: v4: substantially extended and revised versio